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ABSTRACT 

It was hypothesized that a niddle school organization 
embracing grades 5-8 would overcome certain inadequacies of a typical 
elementary an4~ junior high sc^iool organization and would better cope 
with the social and educational needs of older children, 
preadolescents, and early adolescents aged 10-14. Regarding data 
collection, measurement techniques, testing concerns, statistical 
methods, followup studies, attendance figures, teacher attitudes, 
etc., 19 ways were formulated to test the hypothesis. It was 
suggested that the middle school evaluator should first find out 
whether the school is pperating as a middle school. Then the decision 
should be made as to how much of the operation he has the resources 
to evaluate. It should be remembered that good saapling will save 
time and money and will secure further cooperation from those being 
evaluated while providing useful data. Finally measures sjiould be 
gathered over time, sufficient time for the middle schpols to Ijaye 
any impact, and particularly sufficient time to see what happens to 
middle school pupils when they proceed to the next level. 
(Author/RC) 
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More than a decade ago, Cronbac 
equivocal results, I believe that a 



h wrote, **Since group comparisons give 
formal study should be designed primarily 
to determine the post-course perforniance of a v^ell -described group with respect 
to many important objectives and side effects/*^ I do not propose today to 
duplicate the many excellent books, articles, and monographs on evaluation, 
to create a fresh acronym, or to preisent a diagram of Rube Goldberg complexity, 
some of which have moved my colleague. Dr. Robert Soar to comment, 'There is 
less here than meets the eye." Instead, I would like to look back at some 
studies which have been done and to proposal which Or. Alexander and I made 
to the USOE for middle-school evaluation which was never funded to get sug- 
gestions for evaluating the middle school which might be consonant with 
Cronbach ' s recommendat Fon . 

From 1917 to 1921 the late Ellsworth Col lings ran an experiment with a 
project curriculum in a rural McDonald County, Missouri school. ^ The design 
would now be called "quasi-experimental.'* Forty pupils from the experimental 
school were matched vnth an equal number from two control schools operated 
conventionally. Over the four-year period 68 comparisons were made on 
achievement, with all 68 favoring the experimental school. To compare atti- 
tudes toward school. Col lings gathered data on percent of pupils in the 
districts enrolled, percent attending daily, percent tardy, percent truant, 
percent enrolling and remaining the entire year, percent completing eighth 
grade, and percent entering high school. Shift in parent attitudes and be- 
havior were compared from 1917-18 to 1920-21. Data gathered included school 
visits, violation of compulsory school laws, support of a consolidated rural - 
high school, use of school apparatus for testing milk and seed, and use of 
the school 1 ibrary . 

Another basis of comparison was changes in out-of-school activities of 
pupils in the experimental and control schools. These included books read, 
proportion of pup I Is read ing . da F ly newspapers in the home, proportion of 
children taking Instrumental music lessons, proportion participating in 
community activities, proportion carrying on home projects during vacations, 
proportion reading magazines at home, proportion practicing ordinary health 
habits, reduced incidence of common diseases, proportion of children partici- 
pating in parties and games in homes, and proportion of children saving ten 
or more dollars from earnings during the school vacation. 



Parent activities were also studied. These included reading farm journals, 
daily reading of nev/spapers, attendance at night community meetings, partici- 
pation in community activities, testing dairy cows, testing seed corn, shifting 
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to thoroughbred cattle and poultry, change In incidence of disease. Also 
studied were changes in ordinary community conveniences. These included 
subscriptions to papers, farm journals and magnzines, screening, yard 
beaut if icat ion, home playground andi apparatus , modern conveniences in 
the home, and Increase in musical instruments in the home. 

In addition Ceilings collected data from 120 teachers who visited the 
experimenta? school on how they felt about whcit they saw and whether they 
tried the practices in their own schools. 

It should be noted that the experiment ran for four years. Usually, 
Col lings compared changes in the tv/o schools in percents. Today we would 
probably use analysis of variance, sometimes of covariance, chi-square, and 
other nxjre rigorous procedures » However, one can apply a sign test to his 
data, and the odds against the number of differences favoring the experimen- 
tal school by chance are truly astronomical — something less than one in a 
sext i II ion for achievement , alone. 

Of even greater value would have bean finding out what happened to the 
*nOO percent'^ of the eighth graders v/ho went on to high school, and even fur- 
ther, what happened in the years after they left high school? Were they 
identifiably different from pupils who had gone through conventional, sub- 
ject centered rural schools? 

Two other studies from the v;rfter's grab-bag, one small and one large, 
are highly suggestive for the person who is planning evaluation in the middle 
school. One is Harold Fawcett's The Mature of Pro of. 3 The other is The Story 
of the Ei ght-Year Study. Fawcett attempted to teach plane geometry in such a 
way that the logic of deduction, learned in a prejudice-free area, .could be 
deliberately applied to other areas to improve the quality of pupil thinking. 
He used a variety of evaluative devices, including tests of knowledge of 
geometry, but also tests of critical thinking, reports of graduate students 
and visitors to his classes in the Ohio State Laboratory School, anecdotal 
materia) col lected from formal and informal parent conferences, reports from 
other teachers, judgments by teachers in other areas of essays written on the 
nature of proof and anecdotal and interview material from the pupils. Much 
of what Fawcett did might be categorized as a *bn6-shot case study, ^' yet 
his model of using a wide variety of sources for gathering data, all of which 
seemed to point in the same direction — that pupils were using deductive thinking 
more effectively In many out-of-class situations — is suggestive to the person 
v/ho must evaluate an on-going enterprise v/here he had no chance to participate 
from the beginning. 

« 

The Eight-Year Study followed graduates of 30 innovative high schools into 
college and compared their performance v/ith a group tediously matched who were 
graduates of conventional secondary schools. This v/ould exemplify Cronbach's 
call for post-course performance of a well described group. The study would 
be at best associat ional , and has been criticized because matching was used. 
However, in terms of academic achievement and student activities, the experi- 
mental group exceeded their matchees in almost every comparison except foreign 
language achievement and participation in campus religious activities. The 
graduates of the riost radically different schools had wider margins over 
their matchees than did those from the rroderately innovative schools. This 
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study was v/el 1 Financed and had the talents of such distinguished persons as 
Rdlph Tyler conducting the study* It also ran for a suitable long time. 
As a basis for decision making that "progressive*' education v/orkod in the high 
school and should therefore be continued and be extended, the study was al- 
rr.ost ideal. Unfortunately, the results were largely neglected and had little 
impact on the subsequent history of American secondary education. Neverthe- 
less, the middle-school evaluator might still be rewarded by looking back at 
the Eight Year Study. 

Dr. Alexander and I were encouraged to expand a rather irodest middle- 
school evaluation proposal into one v;h?ch would run for nine years. We 
had secured the cooperation of school systems in Montgomery County, Maryland, 
Dade County (Miami, Florida, and Atlanta, Georgia. All three were just be- 
ginning to move toward middle schools. There was even a fair chance that we 
could have had random assignment of pupils to experimental middle schools and 
conventional elementary and junior high schools. 

V/e hypothesized that a middle school organization embracing grades 5 
through 8 would overcome certain inadequacies of a typical elementary and 
junior high school organization and would better cope vnth the social and 
educational needs of older children, preadolescents , and early adolescents 
from 10 to 14» We also felt that there would be a concomitant impact on " 
teachers and parents v/hich might be measurable or observable, V/e proposed 
testing the following hypotheses in the following v/ays: (Throughout v;e 
expected to do much sampling for data gathering.) 

^ • Pupils in the middle school v/ill become more self-directed learners 
( than pupils in the control school s> 

Data would be collected by interview, observation, and case studies. 
It v/ould be expected that Independent study would be used in a 
greater variety of v;ays, by a higher proportion of teachers, and 
involving a higher proportion of students. It v/ould be expected 
that «r,ore time would be spent in libraries, science laboratories, 
language laboratories, etc, and somev/hat less time in group 
instruction in classes for the pupils in the middle school. The 
activities carried on by pupils would be cooperatively planned 
and tailored to the individual pupil and he would be responsible 
for the effective use of large blocks of time with a minimum of 
supervis ion. 

2 • Pupils i n the m iddl e school will have fewer and/or less in tense soc ial 
and ps yc holo gica l pro blem s than pupils in conventional schools , 

Records. will be kept of referrals and diagnoses of problem cases* 
The school systems listed have school psychologists and school 
social workers to deal v/ith cases. Data would be kept at all 
times. In addition, random samples of pupils (stratified by 
sex and grade level) would be drawn each year and given tests 
such as the Rorschach, Thematic Apperception Test, Combs Picture 
Story Test, or other projective instruments. These could be~ " 
analyzed for indications of psychological problems. 
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3 • Ac h j e v<yHr^^^^ .n icid le school pup i Is jon^ . a fT^dnrdj z te sts v^IJ J_ 
ec}u-^l or exceed that oF pupi ls in conventional s choois . 

Pupils would be tested at the same time oF year. Group tests of 
mental ability v/ould also be used. If significant difFerences 
were found in mental age, chronological age, or social-class 
background between pupils in tha experimsntal and control schools, 
analysis of variance and covariance would be used to test the 
hypotheses. 

• twiddle school pupils will achieve as v/ell as or bett er on stan dard 
moasures of phys i ca I f itness and health as pupi ls in conventional 
schools . 

Tests would be selected and used as In (3), except that analysis 
oF variance or t-tests would probably be used*. Suggested tests 
would be the AAHPER Youth Fitness Test and the Health Behavior 
Inventory, 

5 • P_up i 1 > in the midd] e sc[iool wil l hold more favorable attitudes 
tov;ard school and school in g than w i^ 1^1 0un i I s in con vent iona 1 
schools . 

The Cattle Scale, developed by Jean A, Battle during the Kellogg 
Leadership Study at the University of Florida, a SOritem test, 
will be given to pupils. It has been widely used in grades 5 
through 11 . 

^ • Middle scho ol pupFls v;?!! hold more adequate sel f -concepts than 
j//ij I pupi ls in th e co nventional schools . 

Projective techn iques v/i 1 1 be used with random samples of pupils 
from the experimental and control schools. Samples of 10 per 
cent w?n be drawHj strattf ted by sex and grade Jeve), usJng 
tables of random numbers. Care will be taken to avoid '*sel f-report*' 
instruments. 

Data will be gathered annually and v/i II be treated by analysis 
of variance. 

7 • Social acceptance among middl e school pupils vn 1 1 be higher t han 
among tho se i n conve ntional sc hools . 

The Ohio Social Acceptance Scale will be used to test this hypothesis 
Pupils get compos ite scores by all classmates by sex. 

^ • J-hp Avej'acje da t ly attendance reco rd of midd le school pupi Is wi 1 i 
excel that of pupils in conventiona l schoo ls . 

Since participation in the state distribution oF funds depends 
UjX)n meticulous keeping of ADA records, this information will be 
available by grade level for all schools and can be tested for 
differences In percents or proportions. Comparisons will be 
made at the end of each year. 
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9. Heasurcs of creativity among middle school pup ils wi 1 1 show an 
jj3jrr£a5ej^ during midd le school years . 

Measures of creativity tend to decrease during the middle school 
years. It is believed that creativity as measured will remain 
constant or increase. It can be tested in experimental and control 
schools with a test such as the Guilford tests of creativity or 
Torrance^s adaptat ion when more suitable for younger age groups. 

1 0 • Middle school graduates w ill compile bette r a cademic and social 
records In the ninth gr ade than will ninth-graders from the 
control schools. 

This will involve follow-up studies in the ninth grades, 
preferably in the same schools, of pupils who have had one 
year, two years, three years, and four years, rf time permits, 
in the experimental school. Academic records v/ill be deter- 
mined. by grades obtained in required courses and comparisons 
made through analysis of variance. Records of acceptance by peers 
v;ill be measured by a test such as the Ohio Social Acceptance 
Scale and also by checks of participation and leadership in 
nonclass activities such as clubs, dramatics, journalism, and 
the like. Frequency of membership and office held or committee 
memberships and chairmanships will be studied. 

Middle school graduates will drop out less frequently from 
seni or high school than will pupils who follow the traditional 
pattern . 

School laws ordinarily require that reasons why pupils leave 
school before graduation be categorized and recorded. The 
proportions in categories could be compared between students 
from the middle schools and those from the control schools. 

12, Middle school teachers v/ill more often use practices which expert s 
generally recommend as superior . 

The Mathews Teacher Activity Questionnaire is a 90-item scale 
developed and used in the Florida-Kel logg Leadership Project. 
It covers teacher activities v/ith pupils, other teachers, 
principals and supervisors, as members of professional organ- 
izations, with parents and for professional growth. 

^ 3 • Teachers in the middle school will experience a higher degree of 
professional and self-satisfaction than teachers in conventional 
schools . 

A number of instruments could be used here^ One would be The 
Organization Climate Description Q.uest ionnaire, developed by 
Halpin and Croft. Miles has developed several instruments 
which might be used including Hov; It Feels Here, Do's and Dont's, 
System Analysis Quest ionnai re and Problem Anal ys is Questionnaire. 
Still another one which has been widely used in the Florida 
Leadership Studies is Walker *s Teacher Human Relations Scale, an 
instrument designed to assess a teacher's satisfaction with the 
Q human relations aspects of his present job. 
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^ • L^l^J^Jl?A^*^...ii^_Lh?^ n I rid l c s chop 1 i 1 1 , u till ?e^.. a grcader va riety o? 
19^SPJT'H. .nix^LL^-iA^il convef^t lo nal s choois. 

Records of the rn^dfo departriVv^nts would be studied for frequency 
and variety oF uscf. It v/ould be necessary to see chut in general, 
both experimental and control schools had equal accesr. to the 
various media. This information would be collected each year 
during several onc-week periods selected randomly. 

^5. Teacher turnover yn}} be l ess in the niddle school than^ in • 

convent lorial scliools. 

Records would be kept for the period of the study beginnit^g the 
third year. It is anticipated that the middle school v/ould have 
a career appeal which now is found in elementary and high schools 
but v/hich is often lacking in the junior high where teachers are 
eager to move up or down. Hence it is felt that the hypothesis 
would be supported though care would need to be exercised in 
equating other variables such as nge and experience between the 
experfmentai and control groups. 

1 6 . Teachers in the nml dj ^boo L vn 1 1 be more open to change . 

This can be studied witii Duncan's Curriculum Improvement Measure 
which has satisfactory validity and reliability to compare groups 
of 'the size of the faculties in the study. 

^ 7* X?SJlb5Jlj^jL-v^/ir^^ be lv^:>s frequent than in conventional scho ols. 

School records will be t!ie source of data here. Comparisons will 
be made annually by sex and by age groups beti.'een schools. 

^ ^ • Patrons of the middle school y/ill hold more f^ositive a!:titudes 
tov/ord objectives and procedures of the schooi than patrons of 
conventional schools. 



The Parent Attitude Scale, devc^VopQd at the University of Florida, 
or suitable modification of it Can be used lo assess this. It has 
been used with more than 10,000 parents In more than 100 schools 

in studies made at thse University of Florida over the past decade. 

It has also been v/idely used In other states, including a state- 
wide study in Alabama. 

^ 5 • Pri fiCfpals o f ex perimental and contro l schools wi ll have s [m i lar 
operating patterns within ea ch schoo 1 . system. 

Differenres have been found among some schools on soine of the 
hy»-)0 theses to be tested in the present study v/hich are concomi- 
tants of different operating patterns or styles of school 
principals or different organ i 2:at fona I cl i»nate5 within the 
school principals. V/hile the present study is not concerned v/ith 
leadership style as such, tlie behavior of the principals might 
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be an Gyxplanation of contrary results between or among different 
school systems. A number oF instruments havebeen developed 
to get this kind of data including the Principal Behavior Check 
List from the University of Florida, the Leader Behavior Description 
(Questionnaire by Ohio State, the Halpin and Croft instrument 
mentioned in (13)» and the Executive Professional Leadership Scale 
from Harvard. V/hile sample size would probably be too small for 
significance of differences, this v/ould be useful information fn 
case studies of each school. Information would be collected the 
second, third, and alternate years, in the spring. 

Still another source which should not be skipped Is observation. Is the 
middle school doing di fferent things from the traditional elementary or junior 
high school? Observation might be either highly structured, using one of the 
many appropriate systems available or it might be unstructured. Recently I 
visited the Martin Luther King Middle School in Atlanta. Within five minutes 
jthe visitor could see that things were different and that they were v/orking, 
' 'More than a hundred pupils v/ere v/orking on a variety of tasks, most of them 
highly self-directing. Teachers v/ere thare to help as needed. A twelve-year 
old boy was tailoring a sports jacket with great skill • Other boys and girls 
were engaged in other home economics activities. Elsev;here pupils were vjorking 
on mathematics, language arts, reading, social studies, science, and even 
typewriters and calculators. 

This particular group was housed in a structure designed to fit a program* 
The program was in its third year of operation v/ith all kinc|s of resources and 
support personnel. It had been planned. ^ 

This program started in an inner-city school where part of the pupils 
were randomly assigned to DICEP. Other pupils took part in a less planned 
program. At the end of the second year the research assistant in charge of 
evaluation could report: 

During 1972-73, the DICEP pupils scored higher than did the control pupils 
on two criterion referenced tests; the DICEP pupils had a more positive 
rating about themselves, other pup i Is, teachers, school administrators, 
and the school as a. whole than did the control pupils on The Student 
Attitude Scale; the teachers of the DICEP pupils rated them more pos it ively 
than did the teachers of the control group, statistically significant at 
the .01 level, on the Florida Key ; 8k percent of the teachers' responses 
about DICEP were positive on the Teacher's Evaluat ion Scale ; there was 
an average positive response of 75 percent from pupils about DJCEP on the 
Student Evaluation Scale ; and five of the six offices of student govern- 
ment were filled by DICEP pupils, although DICEP pupils comprised less 
than one-fourth of the pupil population. 

In looking to the more recent past, I would change little in rewriting the 
chapter on '^Evaluating the Middle School'* In The Emercient M iddle School^.5 The 
paradigm on multiple criterion measures in evaluation of school nrograms which 
Metfessel and Michael originally presented as an AERA paper in 1967 is reprinted 
in Worthen and Sanders' Educational Evaluation: Theor y and Practice .^ It con- 
tains unusual 1y comprehens ive suggest ions for all kinds of criterion measures. 
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In surrriory, the n^.iddle school cvnluc^tior sivjuid first find out v/hethcr the 
school is operatinn as a middle schoc). Then h:^ sho'.ild decide hov; much of the 
oparcjtion he has the resources to evoluote. He should remomber that good 
sampilny v/ill save tima, n-.ono.y, and uncurf* further cooperation from those 
being cvoluatcd while pruvidincj useful d<.'^ta. Finally, he should gather 
Measures over tlme^ sufficient timiJ for tiie middle school to have an Impact, 
and particularly sufficient t ioK-i to see v/hat happens to middle school pupils 
when tfiey go on to tha next level. 
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